What is calibration in statistics?

Calibration in statistics refers to the process of ensuring that a statistical model is well-calibrated, which means that the predicted probabilities assigned to events are accurate and consistent with the true probabilities of those events occurring. Calibration is important in statistical modeling because it affects the accuracy and reliability of the model's predictions.

One way to assess calibration is through a calibration plot, which compares the predicted probabilities against the observed frequencies of the events. A well-calibrated model should have its predicted probabilities fall close to the 45-degree line on the calibration plot, indicating that the predicted probabilities match the observed frequencies.

Calibration can also be assessed through various statistical tests, such as the Hosmer-Lemeshow goodness-of-fit test or the Brier score, which quantify the degree of calibration of the model.

Calibration is particularly important in predictive modeling, where the goal is to accurately predict the probability of an event occurring, such as predicting the risk of a disease or the likelihood of a customer making a purchase. Without proper calibration, the predictions may be unreliable, leading to poor decision-making and potentially harmful outcomes.